Augmented reality (AR) is a technology combining virtual objects and/or information with perceptions of the physical real-world environment through the use of sensors, computer-generated data, and/or multimedia playback devices in order to enhance the users' perception of the physical real-world environment. For example, a camera can be used to video-capture a scene and a display unit such as video display glasses or goggles is used to project a computer-processed view of the scene to the viewer. Different from virtual reality where the viewer is presented a total artificial computer-generated viewing experience, only certain computer-generated imagery or virtual object is superimposed into the computer-processed view of the scene. The computer-generated imagery or virtual object can be viewer-interactive or driven by real-time user control or pre-configured settings and instructions.
One key measure of the AR technique is how realistic the resulting augmented video as perceived, or how well the integration of the computer-generated imageries or virtual objects with the view of the scene. This requires the correct association of the computer-generated imageries or virtual objects and their placement locations within the scene throughout a motion sequence or consecutive video frames of the scene. One common issue among existing AR techniques is the tracking of three-dimensional coordinates and orientations of two-dimensional planar markers for placements of virtual objects within a video-captured scene. The tracking of markers allows the virtual objects to be superimposed into the scene at the marker locations and orientations through the motion sequence of the scene.
However, the current state of the art does not have a satisfactory technique for addressing the situation in which a tracking marker for virtual object is placed on a deformable surface in a video-captured scene. A deformable surface can be found on a soft article such as a shirt on a person in motion, a flying flag, or moving water surface. A virtual object superimposed on a deformable surface not only need to properly appear at the right position in the right orientation during a motion sequence, its texture and shape must change along with the distortions of the deformable surface.